Ontology-Based Multi-Party Meeting Understanding

This paper describes current and planned research efforts towards developing multimodal discourse understanding for an automated personal office assistant. The research is undertaken as part of a project called The Cognitive Agent that Learns and Organizes (CALO) (see http://www.ai.sri.com/project/CALO). The CALO assistant is intended to aid users both personally and as a group in performing office-related tasks such as coordinating schedules, providing relevant information for completing tasks, making a record of meetings, and assisting in fulfilling decisions. Our focus within this enterprise is on understanding, describing, and automatically participating in multimodal human-human and human-computer discourse amongst CALO users and the system itself. This aspect is functionally realized by the system in its role as a persistent presence before, during, and after meetings; firstly by helping to set up and coordinate meetings and meeting agendas; secondly by extracting detailed information about what was discussed, what the participants’ actions were, and what decisions were reached; thirdly by interactively reporting on this extracted information; and eventually by interactively providing relevant and useful information or responding to queries during the meeting itself. Currently, the system is not interactive during meetings (though this is a plan for future versions), which al-